Business Analyst Community & Resources | Modern Analyst

AI in Testing: Do You Need It? This Article Will Help You Decide!

Rajesh-N — Wed, 28 Oct 2020 07:03:00 GMT

What Everyone Must Know about AI in Testing

Artificial Intelligence is the buzzword that we frequently keep hearing. Its widespread popularity and influence can be understood from the way industries adopting AI in their organization. Whether it’s Healthcare, Automobile, Banking & Financial Services, or Airlines, many industries have started adopting AI.

Artificial Intelligence has not only impacted industries, it has also empowered the SDLC for faster development and deployment. The combination of automation testing and AI has numerous advantages.

Accelerates Timelines

Software testing needs a notable amount of time to validate the quality of an application. As manual testing takes more time, automation testing supports in handling repetitive tasks, with a greater focus on the new implementation to catch the defects faster. Similarly, in addition to automation testing, AI can support automation by creating faster tests (test generators), handling changes (Self-healing tests), and reducing the number of lines of code.

Increases Test Coverage

Automation testing handles repetitive tasks. But with the increase in the number of application technology, test automation tools have difficulty in adapting to those changes. As a result, the test coverage using automation has been moderate for a long time. Due to AI-Powered visual recognition, automation tools are now capable of handling the application, which in turn increases the test coverage.

Improves Accuracy

The quality of the product is ensured through testing. And the accuracy of testing is ensured by QA. But to err is human because even an experienced tester can commit mistakes while performing a repetitive task. Automation testing prevents those mistakes, and an increase in the volume of data for repetitive tasks never impacts the quality of the test. AI-generated data supports creating accurate data and reduces time.

AI and testing have come closer than one can imagine. The implementation of the new features in the existing automation tools or the evolution of new automation tools with AI features are its best proof.

Tricentis – AI Engine NEO
SmartBear – ReadyAPI - AI Driven Test Generation
SmartBear – TestComplete - Hybrid Object Recognition Engine
Parasoft – Smart Generator
Eggplant
Percy
Applitools
Mabl
Functionize
ReportPortal.io
Healenium

Does Artificial Intelligence (AI) Replace Traditional QA Testers?

A lot of thinking persists as to whether AI will replace the QA testers' job. However, that’s not the case. In fact, AI will never replace the role of a tester. Testers will be able to test better with the help of AI-powered tools.

There are few common things between traditional testing and AI,

Traditional Testing:

Prepare Test Cases
Prepare Test Data
Test Execution

AI-Based Software Testing:

Prepare Test Data
Analyze and Clean the Test Data
Train the Test Data by Creating AI Models by Running the Regression Suit

Key Areas of AI in Testing where Traditional QA Testers can work

Regression Optimization:

Modelling the Regression Suit
Solutions for:
- Test Suit Impact for CR
- Patchwork
- Upgrade on the applications
Automated prioritization of test cases/scripts-based on machine learning

Defect Analytics:

AI-based sentiment analytics to find the requirement/positive and negative scenarios for better decision making
Real-Time Dashboard for displaying the current status of the Bug
Analytics-driven workload modelling for
- Defect prediction
- Code coverage
- Response time
- Scalability prediction

Smart Automation Testing:

On every release, find the change detections in the object parameters across the scripts
Self-healing scripts, when there are new CR/Upgrade/Patches
The dashboard on the Requirement Coverage system

Performance Testing:

Workload modelling and Response Times Analysis by ML Modelling
Agent-based System Modelling will be handled for Performance Testing

Conclusion:

Being a QA, either we can use AI for testing, or we can start testing AI. For both of these, we need to understand AI. Let’s explore AI.

This article was originally published here.

ISO 25010

Adriano Comai — Fri, 20 Jan 2012 09:36:00 GMT

ISO 25010, "Systems and software engineering - System and software quality models" was published in March 2011.

The ISO 25010 standard defines 2 broad non-functional categories of requirements, "Quality in use" and "Product quality", articulated in 13 characteristics, many of which are further subdivided into sub-characteristics. This new standard replaces the previous ISO 9126 standard, heavily used both in requirements management and in testing.

To find out more, see my pdf presentation.

Top 10 User Acceptance Testing Preparation Tips

Seilevel — Thu, 05 Aug 2010 18:15:00 GMT

If you are a Product Manager or Business Analyst in charge of managing users through User Acceptance Testing (UAT), here are the top 10 things to do to prepare:

Formal scripts – prepare formal scripts for the business users to run. If you can re-use any of QA’s scripts, all the better. At a minimum, use your use cases to build test scripts. As an added bonus, these scripts will serve as training to the business users on how to use the system after deployment. We suggest you have scripts for testing both functionally and migrated data.
Informal scripts – prepare informal, unstructured scripts for the business users to run as well. I strongly encourage you to do these in addition to formal scripts, in that these are the ones that will pull out defects about how the system isn’t intuitive to use. In addition, they may think to test things you didn’t formally script. As an example, this type of script might simply say “Login to the system and take a training course.” And you are hoping it’s intuitive to the user to figure out how to do that on their own.
Use a tool – we strongly encourage you to put your scripts in a tool and teach the business users how to use that tool. For example, Quality Center is a tool that works well for this.
Master data – create master data that can be used for testing by the business users. This includes logins and passwords and any data they must look at and/or consume in the tool. A great starting place to determine what data you need is to look at your Business Data Diagrams, and then of course look at your scripts. For example, if you have a training system, upload sample training courses for them to take during UAT. You should also organize this master data into a format such as a spreadsheet by test case, so they can quickly reference what data they should use in each script.
UAT Kick-off deck – Create a slide deck to kick the UAT window off with. This kick-off should include the scope of testing, a reminder about the value of the system, a reminder that it is a testing phase and they will find defects in the system, and instructions on how to perform UAT. You need to teach them about using the tools, how to login, and even where to go to access the system.
UAT User Manual – Create a manual for the users to quickly reference to while they execute the UAT scripts. You can hopefully reuse some or all of your kick-off slides. You definitely must include where to access the system (URLs), logins, and where to find master data.
Pre-run scripts – Ideally you should pre-run the scripts before the business users try to execute them. You are familiar with the system, so your eyes on the scripts will be looking for things that are not obvious or incorrect steps. This will help ensure a much more smoothly run UAT.
Teach them how to write a good defect – If you want to avoid a lot of manual labor yourself, teach the business users how to enter their own defects into a defect tracking system (and yes, I’m assuming you have one!). You need to teach them what information to include (logins, urls, steps to recreate) and how to set severity and priority values if appropriate.
Coordinate build schedule with dev – Make sure your dev team is onboard with your UAT testing schedule so that they don’t do a build while users are trying to test. And more importantly, if they do a build overnight, that they don’t take the system down with a broken build! In general you need to coordinate with your entire IT team, I just call this one out as they have an immediate way to cripple testing by accident.
Work with a business owner so they truly own acceptance – All of that said, you need to make sure there is someone in the business who owns the UAT process. You are simply here to facilitate it going well and do a lot of the prep work for them. But truly, they must be the ones who own acceptance of the system or they will never actually adapt it for use. So every step of the way as you go through your prep tasks, be sure you are getting the business UAT owner’s buy-in!

And a bonus #11: Have fun with it! This is when you get to see your system come to life with the people who will be use it. If you have fun, they’ll be more likely to have fun and actually enjoy this!!If you are a Product Manager or Business Analyst in charge of managing users through User Acceptance Testing (UAT), here are the top 10 things to do to prepare:

Formal scripts – prepare formal scripts for the business users to run. If you can re-use any of QA’s scripts, all the better. At a minimum, use your use cases to build test scripts. As an added bonus, these scripts will serve as training to the business users on how to use the system after deployment. We suggest you have scripts for testing both functionally and migrated data.
Informal scripts – prepare informal, unstructured scripts for the business users to run as well. I strongly encourage you to do these in addition to formal scripts, in that these are the ones that will pull out defects about how the system isn’t intuitive to use. In addition, they may think to test things you didn’t formally script. As an example, this type of script might simply say “Login to the system and take a training course.” And you are hoping it’s intuitive to the user to figure out how to do that on their own.
Use a tool – we strongly encourage you to put your scripts in a tool and teach the business users how to use that tool. For example, Quality Center is a tool that works well for this.
Master data – create master data that can be used for testing by the business users. This includes logins and passwords and any data they must look at and/or consume in the tool. A great starting place to determine what data you need is to look at your Business Data Diagrams, and then of course look at your scripts. For example, if you have a training system, upload sample training courses for them to take during UAT. You should also organize this master data into a format such as a spreadsheet by test case, so they can quickly reference what data they should use in each script.
UAT Kick-off deck – Create a slide deck to kick the UAT window off with. This kick-off should include the scope of testing, a reminder about the value of the system, a reminder that it is a testing phase and they will find defects in the system, and instructions on how to perform UAT. You need to teach them about using the tools, how to login, and even where to go to access the system.
UAT User Manual – Create a manual for the users to quickly reference to while they execute the UAT scripts. You can hopefully reuse some or all of your kick-off slides. You definitely must include where to access the system (URLs), logins, and where to find master data.
Pre-run scripts – Ideally you should pre-run the scripts before the business users try to execute them. You are familiar with the system, so your eyes on the scripts will be looking for things that are not obvious or incorrect steps. This will help ensure a much more smoothly run UAT.
Teach them how to write a good defect – If you want to avoid a lot of manual labor yourself, teach the business users how to enter their own defects into a defect tracking system (and yes, I’m assuming you have one!). You need to teach them what information to include (logins, urls, steps to recreate) and how to set severity and priority values if appropriate.
Coordinate build schedule with dev – Make sure your dev team is onboard with your UAT testing schedule so that they don’t do a build while users are trying to test. And more importantly, if they do a build overnight, that they don’t take the system down with a broken build! In general you need to coordinate with your entire IT team, I just call this one out as they have an immediate way to cripple testing by accident.
Work with a business owner so they truly own acceptance – All of that said, you need to make sure there is someone in the business who owns the UAT process. You are simply here to facilitate it going well and do a lot of the prep work for them. But truly, they must be the ones who own acceptance of the system or they will never actually adapt it for use. So every step of the way as you go through your prep tasks, be sure you are getting the business UAT owner’s buy-in!

Do you have comments? Want to check out our other blogs? Check us out here.

Why Business Analysts Should Not Perform Quality Assurance

Jarett Hailes — Wed, 06 Jan 2010 02:33:00 GMT

I seem to find quite a few job postings for Business Analysts that have a couple of lines in the ‘tasks to perform’ description that scare me like:

Define test strategy and test plan
Develop and execute test cases
Perform stress and load test scenarios

It’s not that I don’t believe I am capable of these tasks at some level of competence; after all most BAs have had to perform these activities at some point in their career. I don’t believe a BA should be expected to do Quality Assurance tasks. I can understand the temptation to leverage Business Analysts to perform Quality Assurance tasks, namely:

BAs already have a good understanding of requirements, use cases, etc. so it should be easy for them to come up with testing approaches and test cases
BAs are ‘analytical’ in nature and thus are well suited to QA activities
After the requirements are done BAs just typically sit around or are moved to another project, so we might as well give them something to do so they can be with the project for its entire lifecycle (and be held accountable for ensuring in the end user needs are met)

While these arguments all have some level of credibility, I don’t believe that having BAs perform QA tasks should be considered good practice. Don’t get me wrong, I believe that BAs need to work closely with testers to ensure that requirements are well understood, that requirements can be traced end-to-end, and to help facilitate discussion amongst the team to develop a testing approach. I don’t even mind having to execute test cases if the team’s in a crunch and need some more people to run through scenarios. But overall, I believe that having a Business Analyst being in charge of testing activities (especially if it’s on the same project as doing BA tasks) is not a good idea for the reasons I discuss below.

Quality Assurance is a Separate Role

As the BABOK V2.0 has so eloquently stated, Business Analysis is about working with stakeholders within an organization to identify solutions that will help the organization meet its goals. The BABOK barely mentions testing, only to indicate that a Tester is “response for determining how to verify that the solution meets the requirements defined by the Business Analyst.” These are two distinct roles that while they involve leveraging common knowledge and artefacts (e.g. requirements) they definitively involve disparate activities, techniques, and tools.

Quality Assurance is its Own Profession

Not only is QA its own role, there are organizations that promote the tasks performed by testers as its own profession. There are even separate bodies of knowledge, designations and certifications that have been developed around QA activities. Like any profession it takes time to become adept at performing QA tasks, just as it does to become proficient at BA tasks.

While individuals may develop expertise in both Business Analysis and Quality Assurance over several years (in which case they can perform either role) it is unlikely that most BAs have the level of knowledge to adequately perform QA tasks.

Bias Inherent in Performing Both Roles

For me this is the most important consideration – if you are the Business Analyst (or one of the BAs) involved in the gathering of requirements, you end up developing certain biases based on your knowledge of the business and the problem(s) that are to be solved by the solution being developed. Such biases are rarely at a conscious level, but do occur frequently.

The BA can become so intimate with the requirements that they may not realize when certain assumptions are undocumented, or may unwittingly ignore certain paths or scenarios that need to be tested because they understand the recourses that can occur in those circumstances. In essence the BA may become so ‘in tune’ with the requirements as documented that they may omit or neglect circumstances or exceptions in testing plans that need to be considered. An unbiased third party without such intimate knowledge would have a much easier time to spot any inconsistencies.

Such biases can impact the veracity of testing activities. For example, if the BA starts to write test cases based on what’s in their head rather than what is documented, traceability can become an issue. If the BA omits certain test case scenarios since they know that they are unlikely to occur, are not documented, or the BA already knows how the system will handle the scenario (since they saw it at the last demo), then not only is the current testing phase impacted, regression testing on future changes to the system can become compromised.

Lost Opportunity to Have Another Set of Eyes in the Process

As noted above, a BA can form a biased view of the solution and the project based on their work and job duties. It never hurts to have another brain in the fold that is tasked with, among other things, finding holes in documented processes, looking for requirements that may be under-developed, and generally ensuring that the solution is free from issues that would cause stakeholders to reject it. There are few downsides to including another person given the risks inherent with one person performing both sets of duties.

Get a Tester

Overall, if your organization has the capacity to have dedicated Quality Assurance professionals they will pay for themselves in spades (just as Business Analysts do). Using BAs for testing purposes may seem like a slick thing to do, but in my opinion it is almost never the right choice given the option to go with a separate individual.

What do you think? Leave a comment and let me know your experiences. Have you been a BA who has been forced into QA duties? Do you think there’s no issue with someone doing ‘double duty’?

Test driven development

Craig Brown — Sun, 08 Mar 2009 04:18:00 GMT

Test driven development is not actually a test methodology. It's a design philosophy that says "start with the end in mind" (Thank you Mr Covey.)

Here is a slideshow introducing some of the concepts and ideas within the agile method called Test Driven Development.

Test Driven Development Tutorial: